Stereo vision is an image processing technique which acquires distance information by extracting disparity of a corresponding object from images inputted through two cameras in the same way as a person feels a sense of depth through person's two eyes. The stereo vision is mainly used in systems which recognize an object through images and estimate a distance.
In a related-art method, such a stereo vision algorithm may be normally implemented in the form of software (SW) on a PC or in the form of embedded SW using a mobile CPU such as ARM.
However, there is a limit to implementing in the form of SW to achieve miniaturization and high-speed processing, which are requirements of various applications. Therefore, there is a demand for a stereo vision processing system which is implemented in the form of SoC and thus achieves miniaturization and high-speed processing, and a processing method thereof.
Calibration and rectification of an image inputted through a camera are absolutely necessary for achieving high performance of image processing and machine vision.
Regarding an image inputted through a single camera, distortion caused by a lens is compensated for by changing the location of image data (pixels) using a compensation parameter which is obtained through the calibration process.
In addition, when stereo cameras are used, horizontality between the two cameras should be exactly adjusted to output depth, which is an output of stereo matching, on the highest level. Even if a sophisticated machine is used to mount the cameras, a tolerance is inevitably caused. Therefore, the horizontality between the two cameras may be adjusted using the above-mentioned distortion compensation method.
In the camera distortion compensation process, a location transformation matrix of image data for distortion compensation is calculated, and distortion compensation is performed by applying the transformation matrix to the image obtained by the camera.
Since an exact calculation result is more important than real-time processing, the transformation matrix is calculated and parameters which are the core of the transformation matrix are extracted based on a PC which has high calculation abilities. However, the process of compensating for distortion on the image inputted by the camera should output the result in real time for the sake of subsequent image processing and machine vision operations. To achieve this, distortion compensation calculation is processed in hardware such as a high-performance GPU, FPGA, ASIC, etc.
According to related-art distortion compensation methods, pixel location coordinates for distortion compensation are calculated and data is retrieved from a corresponding location of an image based on the pixel location coordinates and processed. Therefore, the related-art methods have long latency until the inputted image is outputted.
In addition, since it is difficult to know when image distortion compensation calculation should be performed for an image inputted by a camera, the size of a space to store image data inevitably increases.